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Abstract 


This study is based on collaboration between a school and a university on professional development training of 4" 
and 5" grade elementary school teachers in a southeastern state in the USA. The study was three-pronged and 
focused on teacher knowledge, pedagogy, and student achievement. We examined how the building of teacher 
capacity affected the performance of underachieving students in math and literacy. Underachieving students were 
targeted with specific strategies, projects, problems solving stories, self-reflection, and higher-level thinking 
questions. Student performance was measured for literacy achievement, with quantitative and qualitative measures 
used for data collection purposes. Students showed progress over previous years in reading scores. Teachers’ 
growth in knowledge related to the content area was evidenced by the consistently high mean scores. Teachers’ 
self-ratings on the frequency of using the targeted strategies were indirect evidence of their implementation in the 
classroom. The results were positive and showed significant progress over previous years concerning student 
achievement. 


Keywords: literacy instruction, professional development, pedagogy and student-achievement, teacher training on 
site, teacher capacity building 


1. Introduction 


Current standards for student learning have been developed by the Commonwealth of Virginia over the past two 
decades and are inclusive of both content and practice standards; these are called Standards of Learning (The 
Standards & SOL-Based Instructional Resources, 2021). The idea behind the Standards of Learning (SOL) is to 
ensure that all students receive a high-quality education. There are different standards for each subject and grade 
level. The SOLs set goals, tests are administered at the end of each school year to gauge the level to which students 
are achieving them, and school accreditation is based on student test scores. This puts tremendous pressure on 
schools to ensure their students pass the annual SOLs. This pressure affects administrators, teachers, and 
eventually trickles down to students. All schools are required to publish the results of their SOL testing in the form 
of a “Report Card”. The report cards are used to determine school funding, so the stakes involved in SOL testing 
are high. To support educators in their implementation of these standards, schools provide ongoing teacher training 
workshops. As the expectations for instructional practice have been raised, there is a need for more focused 
professional development through which teachers can develop knowledge and pedagogical skills to support the 
implementation of these practices in the classroom. 


Since Every Student Succeeds Act (ESSA) replaced the much-criticized No Child Left Behind Act (NCLB) in 
2015, teacher training has been more instrumental than ever in improving the learning and literacy ability of every 
student. Lack of literacy skills will negatively affect students’ ability to succeed in school which can, in turn, be 
detrimental to their future life. Prior research has concluded that teacher qualification including, level of training 
and teaching experience, has a significant impact on student learning (Mertzler & Woessman, 2012; Rivkin, 
Hanushek, & Kain, 2005). Student achievement is influenced by the capacity and training of the individual 
classroom teacher. 


The consensus view in the literature is that effective teacher learning needs to be on-going, content-specific, and 
practice and teacher inquiry based. This view is supported by the research (Harris & Jones, 2019; Timperley, 
Wilson, Barrar, & Fung, 2007; Darling-Hammond, Wei, Andree, Richardson, & Orphanos, 2009). Teachers 
working in high-needs schools are especially in need of intensive professional development of a long-term 
duration. Collaboration between university faculty and school systems to provide highly engaging teacher training 
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based on the needs assessment of teachers can improve teacher competency in pedagogical knowledge. Competent 
teachers, aware of their impact on student learning, are motivated to strive for excellence in their teaching (Gupta 
& Lee, 2020; Gupta & Kashiri 2007). 


The study was part of collaboration between a school division and an institute of higher education to provide 
teacher professional development programs to build teacher capacity and thereby improve student learning. As 
part of a larger 3-year research project, we conducted a mixed design study to analyze the impact of a yearlong 
teacher training that was implemented in a Title 1 school. The current study is based on the final year’s 
collaboration and focuses on the professional development training of teachers who taught 4" and 5" grade 
students in an elementary school in a southeastern state in the USA. The study was three-pronged and focused on 
teacher knowledge, pedagogy, and student achievement. This professional development program focused on 
assisting teachers in the use of a variety of literacy instructional strategies. It was sustained over time to closely 
examine the impact of on-going teacher training on teacher knowledge and pedagogy which can have long-term 
effects on the math and literacy performance of underachieving students. 


1.1 Conceptual Framework 


Multiple research studies have highlighted the benefits of training targeting the professional development of 
in-service teachers to improve teacher quality (Day, Hadfield, & Kellow, 2002; Neimi, 2015). The quality of 
teaching has a significant impact on the learning of students (Anderson, Heibert, Scott, & Wilkinson, 1985). In 
recent years, the need for robust and systematic teacher preparation and development has gained increasing 
momentum. Scholars agree we need more data driven research for studying professional development (Wayne et 
al., 2008). 


Professional development refers to developing and improving teacher skills so that they can better meet the needs 
of their students. Characteristics of effective professional development have been identified from the literature 
(Desimone, 2009; Rotermund, DeRoche, & Ottem, 2017) and are widely endorsed in the field (Lindmeier et al., 
2020; Main & Pendergast, 2015; Richardson, Miller, & Reinhardt, 2019). The need for professional development 
has gained increasing momentum in the current educational practices, and various research studies have 
highlighted the importance of ongoing training of in-service teachers to improve teacher quality (Day, Hadfield, & 
Kellow, 2002; Niemi, 2015). 


Darling-Hammond, Hyler, and Gardner (2017) conceptualize teacher professional learning as “a product of both 
externally provided and job-embedded activities that have the potential to increase teachers’ knowledge and bring 
about change in their instructional practice in ways that support student learning”. Effective professional 
development is fundamental for teachers to enhance their knowledge and pedagogies and conducting regular needs 
assessments to collect data that guides professional learning is crucial. Ajibade and Bertram (2020) describe the 
perceptions of selected teachers regarding professional learning and development experiences. In a study to build 
teachers’ capacity to integrate science and math content, Aguirre-Munoz Yeter, Loria Garro, and Koca (2020) 
found that the combination of content knowledge and teacher feedback should be targets of extended professional 
learning. 


There is consensus amongst researchers on the core elements that form a model for successful professional 
development (Desimone, 2011). To study effectiveness, we used the conceptual framework with the following 
three essential elements: (1) The professional development increases teachers’ knowledge and skills (2) Teachers 
use their new knowledge, skills to improve the content of their instruction, their approach to pedagogy or both and 
(3) The instructional changes boost their students’ learning. 


2. Methodology 


For this study, researchers used the mixed method evaluation design. The study used both qualitative and 
quantitative data to methods of instruction, student achievement, participant perceptions and experiences. Multiple 
data sources were relied on to address the research questions. These data sources included student reading 
achievement scores, teacher questionnaires, grades that the teachers received in the course, as well as classroom 
observations. The study analyzed data obtained from the teacher participants, classroom observations, and student 
achievement data. The questionnaires and observation instruments were designed to reflect the focus of the study. 
Descriptive qualitative classroom observations were conducted in all 4° and 5" grade classrooms. The student 
achievement data consisted of scores related to reading achievement from the state’s annual Standards of Learning 
(SOL) and Benchmark tests. 


2.1 Research Questions 


Professional Development (PD) is the process by which teachers continually enhance and update their knowledge 
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and skills. The purpose of the study was to further develop knowledge and pedagogical skills of in-service teachers 
which has direct correlation with improved student achievement (Niemi, 2015). Simply conducting PD without 
following up with the participants as to whether effective learning is occurring has little meaning. This prompted 
our first research question below. Knowledge and implementation of the knowledge are at two different levels. We 
needed to follow up by visiting the classrooms for observation to see if the newly acquired learning is being 
reflected in instructional approaches in the actual classroom and whether it is meaningful, which led to the second 
question. The final question assessed how acquired knowledge and skills by teachers affect student learning. The 
guiding questions were based on Gaytan and McEwen (2010) study that PD needs to be meaningful and 
measurable and focuses on student learning outcomes. 


1) How well did the participating teachers master the content and skills covered in the professional development 
activities? 


2) To what extent were the teachers able to implement the content and skills covered during the training in the 4" 
and 5" grade level classrooms? 


3) How did the students in the 4” and 5" grade classrooms perform on measures related to reading achievement? 
2.2 School Demographics and Participants 


The participating school was a Title 1 school with nearly 800 students enrolled in grades PreK -five. Most of the 
students (98%) were African American, and a big part of these students (88%) received free or reduced-price lunch 
(a determining factor in Title 1 eligibility in the USA) which resulted in the school being designated as a Title 
Ischool. The Title 1 program is the largest federally funded educational program in the United States of America 
and provides supplemental funds to school districts to assist those with the highest student concentrations of 
poverty to meet educational goals. Due to the low performance of students on the state’s standardized test scores 
Standards of Learning (SOL), the school had been rated “accredited with warning” for the previous two years. 
Schools with this rating receive extra help to improve their standing by developing a two-year improvement plan 
with the assistance of local agencies. To achieve fully accredited status, an average of 70% of enrolled students 
must pass the SOL tests in each subject area at each grade level. Table 1 below shows the SOL scores from the 3rd 
and Sth grade levels along with subject areas. SOLs are tested at these two grade levels. An examination of the 
scores shows that the criterion was not met for 5" grade students in science. In contrast, ninety percent of 3“ grade 
students received passing scores in science. The criterion was met in all other subjects for both grade levels. In 
general, the results demonstrate a steady increase in scores by year. Nine teachers from grades 4 & 5 participated in 
the study. 


Table 1. Percentage of students by grade level, subject area, and year who passed SOL tests at the local school 
compared to state averages 


Grade/Subject 

Gideon Yrl@school State Avg Yr2@school State Avg Yr3@school State Avg 

English 61 72 64 85 70 77 
Math 71 83 34 78 85 88 

History 74 82 78 87 79 89 

Science 715 82 67 84 90 89 

Grade 5 

English 68 83 64 85 70 85 
Math 47 74 34 78 70 81 

History 65 79 78 87 75 85 

Science 43 80 67 84 40 81 


There have been numerous programs and initiatives implemented by the school over the years to promote 
achievement among students. For example, the school implemented a nationally recognized school-restructuring 
model that incorporates effective principles for school governance, pedagogy, staff development, parental 
involvement, monitoring, and assessment. At about the same time, the school was awarded a grant that 
incorporated interactive technology focused on reading, language arts, and math. 
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2.3 Data Collection 
2.3.1 Intervention 


This study is based on data from the third, which was also the final year of the intervention. Nine teachers from 
grades 4 & 5 participated in the study. Each semester, the university’s College of Education faculty members 
provided various professional development and other activities for the teachers based on areas of need identified 
by the teachers and the school principal (see Table 2). The following section describes the interventions for grades 
4 and 5 where the total number of students was 84 and 80, respectively. 


Most professional development activities were concentrated in reading and language arts. Most of the teachers had 
completed one or more reading instruction courses offered by the university. The objectives are listed in Table 2 
and include lesson framework using preparation, assistance during lesson and reflection after lesson practices, 
individual and group lessons, building background, critical thinking, vocabulary building, writing process, and 
building connections between reading and writing. 


Table 2. Objectives guiding professional development activities in reading 


Use a lesson framework when teaching poor readers and low achievers in elementary, middle school, and high school classrooms. 
Organize their teaching by using the PAR Lesson Framework. 

Use individual and group measures in assessing their pupils. 

Learn and practice ways to assist pupils as they read in every grade level. 

Direct and guide their own pupils in reading fictional stories and content area chapters. 

Determine and build experiential background of all pupils. 

Help pupils to analyze, evaluate, and reflect over what they have read. 


Guide pupils to think critically as they read at all grade levels. 


Oo AmAANI DNA PWN 


Teach pupils to use appropriate study skills in reading. 


a 
oO 


Help pupils develop skill in learning new vocabulary, both in comprehension of vocabulary and in decoding elements of vocabulary. 


— 
= 


Use writing to learn strategies to teach pupils about the process of writing, the products of writing, and the reading/writing connection. 


Learn of the influence of cultural and economic differences in teaching reading to diverse learners in all grades and in special education 


a 
N 


classrooms. 


2.3.2 Building Collaborative Relationships with Teachers 


Throughout the academic year, professors met regularly with the teachers, conducted classroom visits, and 
provided materials. The support was provided from formal classes and workshop sessions to more informal 
one-on-one assistance, advising, and modeling. One of the most important things learned was the importance of 
collaboration. It is vital to understand that any changes or innovations, particularly related to curriculum, pedagogy 
or approaches to learning, need to start with the teacher. We can throw all the latest technology and materials at a 
problem but if teachers aren’t brought in early to the process in a collaborative way, effective change is unlikely to 
occur. Teachers are at the core of what teaching and learning is about in schools and need to have their voices 
heard. Rapport is also extremely important and was developed gradually with the teachers. University faculty 
served as coaches or facilitators rather than as “experts” and provided direct instruction through lectures. Decisions 
were made collaboratively. One of the decisions made by the teachers and the faculty members together was to 
focus on a smaller number of instructional strategies, some of which were applicable to other subject areas, in other 
disciplines as well. The topics or strategies emphasized during professional development were based on an 
informal survey of the teachers’ and administrators’ needs and requests, aligned with the research objectives and in 
the areas of reading, math, and student behavior or discipline. 


To improve content knowledge and pedagogical skills in literacy instruction, faculty members worked with 
teachers in identifying the most effective reading strategies. Criteria used for determining effectiveness were 
objective and included student participation and achievement. There was concerted focus on using higher-order 
questioning strategies to challenge students to think more critically. As a result of previous coursework and 
workshops, teachers were well versed in finding out what students know before a lesson but needed to move 
forward with the “assistance step” to address students having difficulties during lesson. Often their difficulties 
were associated with a lack of study skills or diligence in reading or completing homework assignments. Teachers 
concentrated on motivational techniques and ways to promote student accountability. Finally, as a final step in the 
recursive process, student reflection was guided by more challenging questions such as what they learned and 
critiques of the authors and readings. 
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In mathematics, the faculty members worked closely with the schools’ lead math teacher in the use of test data to 
guide instructional decisions. Together, they disaggregated the data to tailor instruction to meet individual needs. 
The program, “Math Buddies”, facilitated this effort because it began simply and progressed to different levels 
depending on individual student progress. The program did not require mastery at each level as there are links and 
connections to comparable concepts across levels. Other strategies designed to actively engage students included 
the use of manipulatives, calculators, and computer applications. 


Faculty members not only worked with the teachers but targeted interventions for individual students as well. In 
reading, faculty tutored students who were failing or performing below grade level individually and used many of 
the strategies prescribed in the objectives to improve their performance. For example, projects, problem-solving 
stories, and library research were used to actively engage and motivate students. Among students exhibiting 
discipline or behavioral problems, other adults in the community were enlisted to provide support and discuss how 
appropriate behaviors might be reinforced. Faculty members visited parents in their homes, went to the students’ 
little league games to talk to their coaches, and attended their churches to speak to their pastors or Sunday school 
teachers. 


2.4 Data Analysis 


The questionnaire (see Appendix A) was administered to the nine 4” and 5" grade teachers at the end of the school 
year after concluding the year-long professional development intervention. Reflecting back, the researchers agree 
that due to a relatively small number of teachers, it may not have been the best choice to use this instrument, even 
though it was anonymous. The questionnaire contained both a quantitative rating scale and open-ended questions. 
The items on the rating scale corresponded to the content covered in the course as well as the various topics or 
strategies that were covered in the professional development workshops. 


3. Results 


Table 3 below shows the descriptive statistics for mastery rating scale items on how well the strategies were 
mastered. The items are organized by general topics covered in the professional development activities (i.e., 
Reading, Math, Classroom Management, and General Instructional Strategies). 


Table 3. Teachers’ ratings on mastery of strategy use in classroom: Descriptive statistics and percentages by 
response category 


Stand. Not at all Somewhat Very well 
Strategy N = Mean 4 
Dev. 1 3 5 
Reading 
1 PAR lesson Framework 9 444 88 0 0 22.1 11.1 66.7 
2 Anticipation Guide 8 4.88 35 0 0 0 12.5 87.5 
3 Two column note- taking 9 4.56 88 0 0 22.2 0 77.8 
4 PreP strategy 8 4.75 71 0 0 12.5 0 87.5 
5 GIST procedure 9 4.78 67 0 0 11.1 0 88.9 
6 Previewing 9 5.00 .00 0 0 0 0 100 
7 Using “What I Know (WIKA) for previewing 9 5.00 .00 0 0 0 0 100 
8 Directed reading 9 4.78 67 0 0 11.1 0 88.9 
9 Webbing and graphic Organizers 9 5.00 .00 0 0 0 0 100 
10 Problem-solving stories 9 4.33 1.41 11.1 0 11.1 0 77.8 
11 Library research 9 411 1.45 11.1 0 22.2 0 66.7 
Math 
12. Manipulatives/hands-on learning 9 5.00 .00 0 0 0 100 
13. Use of calculators 9 4.78 .67 11.1 0 88.9 
14 Use of computer-based math programs 9 5.00 .00 0 0 0 100 
Classroom Management 
15 Effective discipline 9 4.56 88 0 0 22.2 0 77.8 
16 Group work or peers for academic focus 9 4.56 .88 0 0 22.2 0 77.8 
17 Parental and Community Involvement 9 3.89 1.05 0 0 55.6 0 44.4 
18 Motivational strategies to enhance academicengagement 9 4.78 67 0 0 11.1 0 88.9 
General Strategies 
19 Use of technology in the classroom 9 4.00 1.22 0 11.1 33.3 0 55.6 
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20 Conducting internet searches 8 4.25 1.04 0 0 37.5 0 62.5 
21 Use of inspiration 8 3.75 1.04 0 0 62.5 0 37.5 
22 Cooperative learning 8 4.50 93 0 0 25.0 0 75.0 
23 Literacy across the curriculum 8 4.00 1.07 0 0 50.0 0 50.0 
24 Teacher questioning for critical thinking 9 4.78 .67 0 0 11.1 0 88.9 
25 Student self-reflection questioning 8 3.75 1.04 0 0 62.5 0 37.5 
26 Focus on cultural and diversity issues 9 All 1.05 0 0 44.4 0 55.6 


For the items targeting reading strategies, the mean ratings were consistently high with means across the eleven 
strategies ranging from 4.11 to 5.00 on the 5-point scale. Most of the teachers indicated that the strategies were 
“very well” mastered. Three items had the highest possible mean of 5.00, indicating that all teachers strongly 
agreed that the strategies were very well mastered. These strategies were use of webs / graphic organizers, 
previewing the reading generally, and more specifically, by employing the WIKA (what I know already) 
techniques. The high mean ratings for mastery were impressive but not surprising given that these strategies had 
been emphasized for the prior three years. Lower mean ratings were observed for the two strategies that were 
emphasized during the grant activities. The lowest mean ratings of 4.11 and 4.33 were obtained for incorporating 
library research related to the stories read in class and problem-solving stories. It should be noted that these means 
were low in a relative but not absolute sense. 


The mean ratings were also consistently high for the three strategies targeting instructional strategies in math. 
Means of 5.00 were obtained for two of these strategies: the use of manipulatives or other hands-on learning 
activities, and the use of computer-based math programs. The use of calculators also received a high mean rating of 
4.78. 


According to the teachers, they mastered techniques that incorporate technology and active learning in math 
instruction. In reference to the items related to classroom management, the mean ratings were a bit lower, yet still 
generally high; ranging from 3.89 to 4.78 across the four strategies. The highest ratings were obtained for 
motivational strategies used to enhance academic engagement (M= 4.78). Nearly 90 percent indicated that this 
strategy was “very well” mastered. The lowest mean rating (3.89) was obtained in the parental and community 
involvement category. This item also had the lowest mean rating in previous years, suggesting that the 
implementation of this classroom management strategy remains more challenging for some teachers and requires 
further attention. 


The final category of instructional strategies was more general, and, hence, was applicable to all subject areas. The 
eight items in this category were rated lower than those in content reading and math, with means ranging from 3.75 
to 4.78. The highest mean rating (M= 4.78) was obtained for teacher questioning used to promote critical thinking. 
A high mean rating was also observed for cooperative learning (M=4.50). As in previous years, strategies related to 
technology received somewhat lower ratings, particularly the use of computer programs. Lower ratings were also 
obtained for student self-reflection or questioning (M= 3.75). 


3.1 Frequency of Strategy Use Ratings 


The second assessment scale for obtaining data required teachers to rate how frequently they used the targeted 
strategies. The descriptive statistics for these ratings are presented in Table 4. The pattern of results showed 
slightly lower mean ratings on most items but an almost identical rank ordering of items of mean values. For 
example, the same three strategies in reading received the highest mean ratings for mastery as well as for frequency 
of use (the two items related to previewing and webbing/graphic organizers). Again, the lowest mean ratings in 
reading were observed for problem-solving stories and library research (M’s= 3.86 and 3.22, respectively). 
Overall, the mean ratings ranged from 3.22 to 5.00 for reading strategies, from 4.33 to 4.78 for math strategies, 
from 3.67 to 4.75 for classroom management, and from 3.25 to 4.78 for general instructional strategies. Although 
these means were lower than the mean mastery ratings, they still suggest occasional to frequent use of nearly all the 
targeted strategies. 
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Table 4. Teachers’ ratings on frequency of strategy use in classroom: descriptive statistics and percentages by 
response category 


Item Strategy N Mean Std.Dev. Never Sometimes Freq 
Reading 
1. PAR (Preparation/Assistance/Reflection) lesson 7 471 76 0 0 14.3 0 85.7 
2. Anticipation Guide 8 4.50 93 0 0 25.0 0 75.0 
3. Two column note-taking 8 4.25 1.03 0 0 37.5 0 62.5 
4. PreP (Pre-reading Plan) Strategy 8 4.50 93 0 0 25.0 0 75.0 
Dy GIST (Generating Interaction bet Schemata and Text procedure) 8 4.00 1.07 0 0 50.0 0 50.0 
6. Previewing 9 4.78 67 0 0 11.1 0 88.9 
7. WIKA (What I Know Activity) for Previewing 8 5.00 .00 0 0 0 0 100 
8. Directed Reading 8 4.50 93 0 0 25.0 0 75.0 
9. Webbing & Graphic Organizers 9 4.78 67 0 0 11.1 0 88.9 
10. Problem Solving Stories 7 3.86 1.57 0 0 28.6 0 57.1 
11. _ Library Research on Topic 9 «3,22 1.56 0 0 44.4 0 33.3 
Math 
12. | Manipulatives / hands-on learning 9 4.78 67 0 0 11.1 0 88.9 
13. Use of calculators 9 433 1.00 0 33.3 0 66.7 
14. Use of computer-based math program 8 4.50 93 0 0 25.0 0 75.0 
Classroom Management 
15. Effective discipline 8 4.75 71 0 0 12.5 0 87.5 
16. Group work or peer work 9 4.33 1.00 0 0 33.3 0 66.7 
17. Parental and Community Involvement 9 3.67 1.00 0 0 66.7 0 33.3 
18. _ Motivational Strategies to Enhance Academic Engagement 9 4.56 88 0 0 22.2 0 77.8 
General Strategies 
19. Use of technology in classroom 9 3.89 1.05 0 0 55.6 0 444 
20. Conducting internet searches 8 Ce) 1.03 0 0 62.5 0 37.5 
21. Use of Inspiration Software 7 3.86 1.07 0 0 SA 0 42.9 
22. Cooperative Learning 8 4.50 193 0 0 25.0 0 75.0 
23. Literacy Across the Curriculum 8 3.50 1.41 12.5 0 50.0 0 37.5 
24. Teacher questioning for critical thinking 9 4.78 67 0 0 11.1 0 88.9 
25. Student Self-Reflection / Questioning 8 3.25. 1.28 12.5 0 62.5 0 25.0 
26. _ Focus on Cultural and Diversity Issues 9 3.89 1.45 11.1 0 33.3 0 55.6 


3.2 Open-Ended Items 


Close examination of the data showed that eight of the nine teachers responded to the open-ended question that 
asked the participating teachers to describe how the grant activities shaped their classroom teaching. Three of the 
teachers appreciated the array of effective strategies, particularly in reading, that they could employ in their 
classrooms. One teacher commented, “The activities made me aware of the different reading strategies and helped 
me focus on implementing them regularly.” Another said, “the program has provided training and some classroom 
modeling which in turn has made me more aware of reading techniques and reading strategies.” Two of the 
teachers mentioned overall benefits to students in terms of “enhanced learning” and “broadening their classroom 
experiences.” Two teachers noted student benefits associated with specific strategies, stating, “Using the 
anticipation guide proved helpful to get the students ready to read and to give a purpose” and Graphic organizers 
enable students to “make connections”. The second open-ended question asked how their interactions with 
students had changed. Seven teachers responded to this question. Three described students as more motivated and 
engaged, with one teacher saying, “My students were highly motivated and eager to participate.” A theme of 
independence or more student autonomy emerged from two of the responses: “Sometimes the students become 
their own teachers, and the classroom teacher just troubleshoots problems or gaps in the learning process.” One 
teacher described the student/ teacher relationship as stronger using strategies, while another said that interactions 
with students had not changed but depended on who was present or absent. 


Data analysis further showed that seven of the participating teachers responded with their thoughts when asked 
what other kinds of activities would be beneficial for improving instruction and student achievement. One theme 
that emerged pertained to having more time, space, and latitude to expand lessons with one teacher wanting space 
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to move and rotate to centers. More open-ended class time to expand the lesson or to implement more of the 
targeted strategies was also suggested. Another teacher wanted an extended day in math and reading tutorials. In 
reference to timing, one respondent suggested that the program be implemented prior to the beginning of the 
school year. Additional comments focused on the desire for more technology, group activities, and manipulatives. 


3.3 Observation of Fourth and Fifth Grade Classrooms 


All the participating teachers were observed two to three times during the entire training. The lead evaluator and 
principal investigator conducted a total of 21 observations of 30 minutes each. These observations were made at 
the end of the school year because the SOL tests had been administered and the focus of instruction had shifted to 
instruction rather than test preparation. Researchers were able to observe the 4" and Sth grade classes mainly 
during the reading/ language arts block in the mornings to further increase the likelihood that targeted strategies 
would occur. However, we also conducted observations when other subject areas were being covered. The 
evaluator and principal investigator established inter-observer reliability in the same way by observing and 
recording the participants’ teaching strategies using an observation form which appears in Appendix B. 


Observers recorded the demographic information about each teacher, which included teacher’s name, grade level, 
subject area(s), and beginning and ending times. Observers provided a general description of classroom instruction 
and activities. The questionnaire also presented a checklist of strategies with space to write open-ended comments 
about how the strategies were implemented, student reactions, and other contextual information. These strategies 
were organized into four categories: (1) general instructional, (2) reading, (3) math, and (4) classroom 
management. The final section prompted observers to provide additional notes and comments, including any 
conversations with teachers about what was observed. Any other additional course handouts or other relevant 
documents distributed to the students during the observation periods were collected by the researchers. 


Table 5 presents the most frequently observed instructional strategies organized into the three categories. Only 
those strategies that were observed at least three times in different class periods were included. 


Table 5. Most frequently observed instructional strategies by category 


Category/ Strategy Frequency 


General Instructional Strategies 


Student self-reflection/ questioning 
Cooperative learning 
Focus on higher level thinking 


Literacy across the curriculum 


kha AN ow 


Teacher questioning for critical thinking 
Reading 

Webbing and graphic organizers 

GIST Procedure 

Classroom Management 


& oC 


Reinforcement of on-task behavior 
Redirection to academic task 

Correction of off-task, disruptive behavior 
Posted charts or other records 

Other disciplinary strategies 

Focus on higher level thinking 


Literacy across the curriculum 


ROA DWDwWwW ww RN NA 


Teacher questioning for critical thinking 


Teachers employed a wide array of effective general strategies applicable to any content area. The most frequently 
observed instructional strategy was self-reflection/questioning, which was a byproduct of the bio poems, a reading 
strategy, for which students were instructed to ask themselves a series of questions. In another case, the 
self-questioning was prompted by having students generate questions to ask themselves while reading a story. 
Cooperative learning was also frequently observed. 


Students often worked in groups or dyads to complete classroom assignments. The dyads were sometimes formed 
for the purpose of having one student tutor or help other students. In some classrooms, the desks were arranged in 
clusters, comprising groups that regularly worked together. It appeared that students were accustomed to working 
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together in class because they were mostly well behaved and followed instructions. A focus on higher-level 
thinking and teacher questioning for critical thinking often went hand in hand and were frequently observed. 
Examples of these strategies occurred when students were asked to generate steps in a process (“how to make a 
sandwich’). An initial step in the process was the use of brainstorming techniques guided by teacher questions. 
These techniques were geared towards groups as well as individual students. One teacher asked a student to use the 
story to help define and comprehend a word that appeared on a story map. Literacy was incorporated across the 
curriculum. The stories were related to history or science, and poetry was covered in the development of bio poems 
or geo poems (see geo-poem example below). 


Example of a geo-poem: 

Virginia (name of state/city or location) 

Birthplace of the nation (known fact about the location) 

Mother of 8 Presidents (known fact about the location) 

Named after Elizabeth I, the “Virgin Queen” (known fact about the location) 
One of the first 13 original colonies (known fact about the location) 

Site of final Revolutionary War in Yorktown (known fact about the location) 
Old Dominion (another name / nickname) 

Example of a bio-poem: 

Keisha (First Name - line 1) 

kind, honest, sincere, punctual (four words that describe the character - line 2) 
lover of dogs, movies, travel (three words - line 3) 

believes in humanity (one idea or concept - line 4) 

wants peace, love and justice (three things - line 5) 

afraid of night, bugs and death (three things - line 6) 

gives joy and gifts (two things - line 7) 

Watson (Last name / surname - last line) 


By far, the most common strategies observed in reading or language arts were webbing and graphic organizers. 
During the period of our observations, 4"" grade students were writing bio poems that incorporated graphic 
organizers. As noted, self-reflection and questioning were also required in this activity. At one stage students were 
asked to find adjectives with first letters that corresponded to letters in their names. Students used dictionaries to 
help them identify the best fitting adjectives for a particular letter. These adjectives were then transferred to a 
diagram and incorporated into the poems. Webbing and graphic organizers were also used for other activities. In 
one classroom, students were working on “word maps” which required them to provide definitions, synonyms, and 
antonyms. In another, the teacher used a flow map to diagram the steps in a process. 


Example of Webbing: 
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Example of Graphic Organizer: 


aa 
. Charlotte’s 

ra ped 
an ee CE 
Loco nandh Wilbur e Templeton 
ae ie % 


ingebligort —_Ronest 


a 


Example of Word Map: 


The second reading strategy commonly observed was the GIST (Generating Interactions between Schemata and 
Text) procedure particularly used with expository texts though it can work with all types of texts, narrative, fiction, 
non-fiction (Cunningham, 1982). The text is divided into sections that serve as stopping points for pausing to 
clarify and write the main point. GIST helps students get to the main idea quickly and efficiently. This guides 
students in summarizing the stories in their own words. It is important to note that the general strategies were 
primarily observed in the context of reading or language arts, and often these general strategies were implemented 
in conjunction with the reading techniques. 


The most frequently occurring strategies observed related to classroom management were reinforcement of 
on-task behaviors and redirection to academic tasks. Teachers would praise students for their academic 
engagement. The teachers who used the reinforcement techniques tended to have a very good rapport with their 
students. Many teachers relied on posted charts, lists of goals, and other records. There was overlap across 
techniques for rewarding students because students could earn points or extra minutes of recess based on a type of 
token economy system that was posted and recorded. In some classes, teachers referred to posted goals to remind 
students about codes of behavior. Other disciplinary techniques, that were essentially punishers, included calling 
the parents or sending students to the principals’ office. In these classrooms, the students’ behavior was not well 
managed, and less time was devoted to instruction. Overall, classes tended to be well managed, but there was some 
variation across individual teachers. On one end of the continuum, mutual respect among teachers and students 
was evident, and little formal management was necessary to academically engage students. On the other end of the 
continuum, little academic instruction occurred because students were wandering around and talking despite the 
teachers’ effort to control them. 
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3.4 Fourth and Fifth Grade Student Achievement 


To address the third research question related to student achievement, we focused on standardized test scores 
related to reading at the 4" and 5" grade levels. We analyzed 4" and 5" grade benchmark data, and 5" grade SOL 
scores. 


Benchmark scores. One of the data points was the achievement scores of students. For this purpose, Benchmark 
scores were used to measure 4" grade students’ progress in reading as well as editing skills and 5" grade students’ 
progress in reading and writing. The benchmark tests are aligned with the state required curriculum for SOL’s, and 
the assessments are patterned after the SOL tests. The benchmark tests are developed by the state to provide early 
diagnostic information and practice for the SOL’s. The scores obtained from the Benchmark tests were compared 
from the 1“ nine weeks of the school year with those obtained in the 3" nine weeks. Even though this was a 
relatively short time length, it was another source of evidence pertaining to ongoing student development in the 
areas of reading and writing. 


The following table provides the average percentage of points obtained for the reading and editing Benchmark test 
for 4" graders and on the reading and writing benchmark tests for 5" graders. We assigned the earlier 
administration of the tests as the pretest and the late administration as the post-tests. The table shows only those 
students with scores available for both the pre- and post-tests in the analyses. 


Table 6. Descriptive statistics for pre and post benchmark scores by grade level 


Grade Pre-test Post-test 
4" (N=84) Average Standard Dev Average Standard Dev 
Reading 64.73 15.31 74.97 14.85 
Editing 58.05 17.81 67.35 20.99 
5" (N=80) 
Reading 65.27 16.17 72.70 17.66 
Editing 59.48 16.90 63.39 22.13 


Among 4" graders statistically significant improvements were found on both reading and editing scores. It can be 
noted that there was a significant increase in the average percentage of points obtained when comparing pre- and 
post-test scores in reading (t (1,83) = 5.44, p<.001). Students achieved an average of 64.73 percent of the points 
correct on the pretest and an average of 74.97 percent on the post-test. This represents an impressive increase of 
over 10 percentage points. Similarly, we found a significant increase in 4" graders editing scores (t (1,83) = 4.23, p 
= .001. Editing scores improved from a mean of 58.05 to 67.35, reflecting a substantial increase of nearly 10 
points. Analysis of the data shows that 4" graders scored well on these tests, and their scores were improving. 


Turning to the performance of 5" graders on the Benchmark tests, we found improvements in both reading and 
writing scores, but only the change in reading scores was statistically significant (t (1, 79) = 3.21, p =.02). Reading 
scores improved from a mean percentage of 65.27 to 72.70. Although the mean difference in writing scores did not 
reach statistical significance, it did approach this criterion (p= .10) and writing scores did improve from pre to post 
testing. The average pretest score was 59.48%, and the average post-test score was 63.39%. In both areas these 
scores reflect improvement and bode well for the actual SOL results presented in the next section. 


SOL Scores. The SOL data on 5" grade achievement in English (reading and language arts) and writing was 
available for this report. After two years of approaching the statewide benchmark goal of a 70 percent passing rate, 
5" graders have now achieved this rate. Researchers also studied how 5" grade students scored on the English test 
across three years (see Table 7) as a comparison over a period. An examination of the percentages reveals virtually 
no change in English scores from year 2 to year 3, but an increase to 70 percent in year 4. Since the state averages 
have remained relatively constant, the students were steadily closing the achievement gap in English. 


Table 7. Percentage of 5" grade students who passed SOL English tests compared to state averages 


Year 1 Year 2 Year 3 
State Avg School State Avg School State Avg School 
English 83 68 85 66 85 70 
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A breakdown of scores by category provides a more in-depth look at students’ strengths and weaknesses on this 
assessment. Table 8 presents data on the percentage of students by proficiency level on each of the three scales or 
categories that comprise the English test. The results indicate that 5" graders scored highest on their understanding 
of literature elements; seventy-four percent of students scored at the proficient level in this category. Over 70 
percent of students also scored at the proficient level on understanding a variety of resource materials. The weakest 
area was use of word analysis strategies. Only 66 percent of the students scored proficient in this category. These 
trends have implications for instructional design. The data point to the need for more strategy instruction or 
practice in how to analyze words. 


Table 8. Percentage of 5" grade SOL English scores by category and proficiency level 


Category Proficient Not Proficient 
Use word analysis strategies 66 34 
Understand a variety of resource materials 71 29 
Understand elements of literature 74 26 


In reference to writing performance, SOL scores show steady and impressive progress across years. Table 9 
presents the percentage of 5th grade students who passed the SOL writing test during the three-year duration of the 
study. The percentages of students have increased from a low score of 60 percent passing to a high of 93 percent. In 
the last two years the scores were well above state benchmarks. 


Table 9. Percentage of 5th grade students who passed SOL writing tests across four years 


Year Percentage Passing 


Year | 68 
Year 2 60 
Year 3 84 
Year 4 93 


4. Discussion 


Teachers were generally positive about the content and relevance of the academic curriculum and its effectiveness 
in preparing them to teach reading. In response to the first research question, teachers reported that the content and 
skills were very well mastered. This is evidenced by the consistently high mean scores obtained from the rating 
scale items pertaining to mastery on the questionnaires. This trend was true across the categories of reading, math, 
classroom management, and more general strategies. Some exceptions to the pattern of high ratings were the 
relatively lower ratings found on parental and community involvement, the use of software, and student 
self-reflection or questioning. The tendency of teachers to provide high ratings on mastery of content and strategies 
was apparent across the life of the grant. However, teachers’ confidence in their knowledge of the strategies does 
not always translate into their frequent use in the classroom. At the start of the study, participating teachers 
demonstrated a range of understanding with respect to the content focused on professional development training. 
They were all at different levels in terms of their knowledge and skills. An additional layer to these differences 
amongst teachers was with respect to their background experiences and expectations from the training program, 
making it critical to assess teacher understanding of subject matter before deciding on the topics for the training 
workshops so that the workshop curriculum could be tailored to their exact needs. 


In response to the second research question about the implementation of content and skills in the classroom, 
various responses were obtained. Teachers’ own ratings of the frequency with which they used the targeted 
strategies provided indirect evidence of classroom implementation. High mean ratings were obtained on the use of 
strategies in the areas of reading, math, and classroom management. In these areas, we found teachers used 
problem solving stories, library research for stories, and parental and community involvement less frequently than 
the other strategies. Overall, the mean ratings in the general strategies’ category tended to be slightly lower. 
Teachers reported less use of technology, literacy across the curriculum, student self-reflection, and a focus on 
cultural or diversity issues. Participating teachers were able to assess themselves as reflected by the self-ratings 
about how frequently they implemented the strategies in the classrooms that they learnt during the training. Even 
though these ratings were low in a relative sense, in an absolute sense, teachers still reported using these strategies 
at least occasionally. Therefore, based on teachers’ self-reports, most content and skills covered were commonly 
implemented in the classroom. 
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There was additional evidence about the extent of implementation which was obtained from observations in 4" and 
5" grade classrooms. In the area of reading strategies, we observed the use of webbing and graphic organizers the 
most. Student self-reflection, cooperative learning, and a focus on higher-level thinking were among the other 
frequently used general strategies. With respect to classroom management, we mostly observed more positive 
discipline strategies such as reinforcement of appropriate behaviors or redirection to academic tasks. However, in 
some cases classes were not well managed and students were disruptive, which detracted from the time that could 
be spent on academic tasks. 


The third and final question that the study focused on was student achievement. The results were positive and 
showed progress over previous years. This was the first year that 5" grade scores in English reached the state 
benchmark of a 70 percent passing rate. The 5" grade SOL writing scores were even more impressive. Overall, 93 
percent of these students passed the SOL writing test. In addition, the benchmark data revealed improvements 
during the year in reading, editing, and writing scores for both 4" and 5" grade students. For three of the four pre 
and posttest comparisons, the gain in scores was statistically significant. 


5. Recommendations and Limitations 


This study demonstrated the positive effects of a long-term teacher training on enhancing teacher knowledge and 
quality of teaching skills along with improved student learning outcomes. To ensure positive and effective 
long-term professional development activities, selected recommendations are made for high-need schools. The 
first is to incorporate technology applications in classroom instruction. We rarely observed any classroom 
instruction that relied on web-based and interactive technology applications for either the delivery of instruction or 
as a way for students to learn and extend the content. The one exception was the use of software to teach 
mathematics. Questionnaire results also pointed to technology as a competency or strategy in need of 
improvement. As Internet and technology are essential components of life, teacher training should include 
innovative online instruction using various applications. Workshops could focus on training in the use of 
technology and online resources not just in instructional delivery, but also as a learning tool for students to better 
comprehend and apply the material. This would also benefit students by promoting technological skills while they 
study other content areas. Of course, the resources necessary to equip teachers and students with technological 
tools are a prerequisite, and this requires substantial funding. 


A second recommendation is to provide more expertise and training related to classroom management. As reported 
earlier, we saw some variation with the ways in which teachers managed classrooms and student behavior. Faculty 
members’ efforts in targeting more individual interventions for students seem promising. More teacher, parent, and 
even student workshops or other programs could continue to address “effective discipline” strategies and policies. 


Our findings suggest that the level of parental involvement could be increased. Increased parental involvement 
may promote better student conduct, achievement motivation and academic engagement among students. Teachers 
rated the item on parental and community involvement lower than other items under classroom management. In 
addition, we did not observe use of parental involvement activities. The lack of parental involvement is certainly 
not unique to the subject school. It is a common problem among Title I schools, but one that should be addressed. 
Efforts to enhance parental participation might include more of the same strategies implemented this year as well 
as take home assignments that might encourage more parental engagement in students’ academic work (e.g., parent 
signatures on homework or joint projects). Finally, we suggest incorporating the most effective activities 
sponsored by the grant into the school improvement document. This would help build capacity to focus on and 
support the implementation of acquired strategies internally post-intervention. For example, school benchmarks 
and indicators might specify the reading strategies that will be implemented in the classroom to promote continued 
progress in student achievement. 


While the study provided potentially useful findings regarding content-focused professional development, the 
limitations of this study are worth noting. First, the sample size was limited to nine teachers. Due to a relatively 
small sample, it may not be a good choice to use a questionnaire as the instrument, despite keeping it anonymous. 
Second, the school site had simultaneous multiple interventions in progress which made it difficult to isolate the 
effect of focused single intervention. Research in school settings often precludes the control of variables to isolate 
cause-and-effect inferences. Third, this study relied on descriptive, comparative data, with some findings based on 
teachers’ self-reporting which may not always be completely accurate. A fourth limitation of this study was that all 
achievement measures were not administered pre and post for all grade levels to compare with those grades that 
didn’t participate in the study at all. That would have provided us a framework of reference. In addition, due to 
limited number of classroom observations during the instruction time in school, there may have been missed 
opportunities to see how teachers were applying the skills and strategies targeted by the intervention. By 
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conducting observations towards the end of the school year (just before the administration of final tests) 
researchers may not have collected the data that is fully representative of the classroom practices throughout the 
school year. 
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Appendix A 


As participants in the research study activities, we need your feedback. Rest assured that your responses will 
remain confidential. So please be candid. We simply want some summary information to help us evaluate the 
success of the grant activities. The feedback provided on this form will be used as part of larger evaluation 
conducted on the research study activities. 


A list of strategies targeted by the grant appears below. The first section asks you to rate how well you mastered the 
strategies. The second asks you to rate how frequently you use these strategies in your own classroom. Simply 
check the box that best represents your response. 


How well mastered? How frequently used? 


Not 
Strategies Very . 
at Somewhat tl Never Sometimes Frequently 

we 
all 


Reading 
. PAR lesson framework 


. Anticipation Guide 


. Two column note-taking 


. PreP strategy 


. Previewing 


. Using “What I know” (WIKA) for preview ing 


. Directed reading/ thinking 


1 
2 
3 
4 
5. GIST procedure 
6 
7 
8 
9 


. Webbing and graphic organizers 


0.Problem-solving stories 


1. Library research for stories 


Math 


2. Manipulatives/ hands on learning 


3. Use of calculators 


4. Use of computer-based math programs 


Classroom Management 


5. Effective discipline 


6. Using group work or peers to focus on academic tasks 


7. Parental and community 


involvement 


8. Motivational strategies to enhance academic 


engagement 


General Instructional Strategies 


9. Use of technology in the classroom 


20. Conducting internet searches 


21. Use of Inspiration software 


22. Cooperative learning 


23. Literacy across the curriculum 
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24. Teacher questioning techniques to encourage critical 


thinking 


25. Student self-reflection, Questioning 


26. Focus on cultural and diversity 


issues 


Open-ended items (use back of page if necessary) 
1) How have the research study activities influenced your classroom teaching? 
2) How have your interactions with your students changed? 


What other kinds of activities would be beneficial for improving your instruction and student 
achievement? 


Appendix B 
Observation Forms 
Teacher: Grade: Subject area: 
Date/Time | 
Duration: General description of classroom instruction, activities: 


Observation Checklist and Description 


General Instructional Strategies 


Strategy Description 


Focus on higher-level thinking 


Cultural and diversity issues 


Literacy across curriculum 


Use of technology (Inspiration) 


Teacher questioning for critical thinking 


163 


ies.ccsenet.org 


Student self-reflection/questioning 
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Cooperative learning 
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Reading Strategies 


Strategy 


PAR lesson framework 


Anticipation guide 

Two column note-taking 
PreP strategy 

GIST procedure 


Previewing 


Description 


Using “What I Know Sheets to teach” 
(WIK.A) to teach previewing 
Directed reading/thinking 


Webbing and graphic organizers 


Problem-solving stories 


Library research for stories 


Math Strategies 


Strategy 


Manipulatives/ hands on learning 


Use of calculators 


Description 


Use of math software 


Classroom Management 


Strategy 


Reinforcement of on-task behavior 


Description 
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Redirection to academic task 


Correction of off-task/disruptive behavior 


Other disciplinary strategies 


Posted charts or other records 


Additional comments, notes (including notes about conversations with teachers): 


Copyrights 
Copyright for this article is retained by the author(s), with first publication rights granted to the journal. 


This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution 
license (http://creativecommons.org/licenses/by/4.0/). 
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